Papers with partial-input baseline
Partial-input baselines show that NLI models can ignore context, but they don’t. (2022.naacl-main)
Copied to clipboard
| Challenge: | Researchers have shown that many datasets contain statistical biases, or "annotation artifacts" that systems leverage to correctly predict entailment. |
| Approach: | They propose to use edited contexts to examine RoBERTa models' sensitivity to edited context to examine their model's sensitivity. |
| Outcome: | The proposed model can learn to condition on context, despite being trained on artifact-ridden datasets. |
Misleading Failures of Partial-input Baselines (P19-1)
Copied to clipboard
| Challenge: | Recent work establishes dataset difficulty and removes annotation artifacts via partial-input baselines. |
| Approach: | They propose to use partial-input baselines to establish dataset difficulty . they show how trivial patterns only visible in the full input can evade partial-output baseline . |
| Outcome: | The proposed model can solve 15% of previously-thought "hard" examples. |